This invention relates to an image processing apparatus, and more particularly to an image processing apparatus suitable for use for production of a color image signal of a wide dynamic range from an image signal acquired, for example, using a CCD image sensor of the single plate type or the like.
A solid-state image pickup device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) is widely utilized in an image pickup apparatus such as a video camera or a digital still camera, a part inspection apparatus in the field of Factory Automation and optical measuring instruments such as an electronic endoscope in the field of Medical Electronics.
Usually, since a solid-state image pickup device itself can have a single spectral sensitivity, if it is used to pick up an image, then only an image of a single spectral sensitivity, that is, only a monochromatic image is obtained. Therefore, in order to obtain a color image using a single solid-state image pickup device, a conventional method is used wherein an image is picked up with different spectral sensitivities for each pixel, that is, with different colors for each pixel.
As a method for changing the color for each pixel, color filters are used. For example, one of filters of three different colors of Red (R), Green (G), and Blue (B) is used to cover light receiving elements arranged on an image pickup plane of the solid-state image pickup device. Consequently, since the pixels of an image picked up have only one kind of color component, the picked up image is an image like a mosaic with regard to the color (such an image is hereinafter referred to as a color mosaic image).
A technique has conventionally been developed wherein a predetermined image process is applied to color mosaic images obtained in such a manner as described above to produce an image wherein all pixels have components of R, G and B.
It is to be noted that the combination of colors of filters for covering light receiving elements arranged on an image pickup plane of a solid-state image pickup device is not limited to the three colors of R, G and B, butcan include, for example, four colors of G, C (cyan), M (magenta) and Y (yellow).
In such an image process as described above, an influence on the picture quality can arise from the fact that the sampling frequency is different among different colors matters. For example, where a color mosaic arrangement is a Bayer arrangement, the sampling frequency for R or B is ½ the sampling frequency for G. It is to be noted that, although it is possible to make the sampling frequencies for the different colors equal to each other, it is impossible to synchronize the phases of them with each other.
Where the sampling frequencies for the different colors are different in this manner, phase differences appear among different color components restored by an image process and are observed as a color moire on an output image, and are therefore a significant cause of deterioration of the picture quality.
Conventionally, techniques of an image process for suppressing appearance of such a color moire as described above have been proposed. Such techniques are described below.
A first related-art technique includes a method of interpolating colors such that a local ratio of colors may be maintained. According to the method, it is assumed that, in a local region, the direction of an object color of a subject does not exhibit a great variation. The direction of an object color is described in a ratio of colors. For example, it is assumed that the ratios of R and G at a point x and at another point y neighboring each other are equal to each other. In other words, it is assumed that R(x)/G(x)=R(y)/G(y). If G(x) at the point x is known, then R(x) can be calculated if the ratio R(y)/G(y) of the colors at the neighboring point y is acquired. Where the ratio of colors does not exhibit a great variation within a local region, a known average value (R/G)L of the R/G ratio of the pixels in the local region can be used in place of the color ratio R(y)/G(y) at the neighboring point y. In other words, R(x)=G(x)·(R/G)L. Such a first related-art technique as just described is disclosed in the official gazette of Japanese Patent Laid-Open No. Sho 61-501424.
A second related-art technique includes a method of interpolating colors such that a ratio of low frequency components of different colors may be maintained. According to the method, (R/G)L calculated in the first related-art technique described above is approximated using the ratio RL/GL of a low frequency component RL of R and a low frequency component GL of G. It is to be noted that, since the low frequency component RL of R and the low frequency component GL of G can be calculated independently of each other even if the same pixel does not have both components of R and G, it is not necessary to perform interpolation for G first in order to calculate the ratio RL/GL.
In the first and second related-art techniques described above, the arrangement of a color on pixels of a color mosaic image (the arrangement is hereinafter referred to as a color mosaic arrangement), and the first and second related-art techniques have a problem to be solved in that, for a color mosaic image of an arbitrary color mosaic arrangement, it is impossible to effectively suppress the appearance of a color moire.